How To Write An Online Poker Bot: Part 1
The Other Toke
----------------------------------------
INTRODUCTION
Recently many questions have been asked about the feasibility of
implementing an online poker "bot". For those few of you unfamiliar
with the concept, a poker "bot" is a piece of software with the
intelligence to play a (presumably winning) game of online poker, and
to do so without human guidance. An effective bot could be instructed
to play for hours or days at a time, garnering a steady profit, while
the user went about his business. The benefits of a bot are thus
twofold: making money on the one hand, and saving time on the other.
This article is an informal and somewhat unorganized description of
techniques that might be used to implement such a bot, delivered in
two parts. My aim in writing is to inform honest players, who happen
not to be programmers, about what is possible and how it might be
implemented, and also to get some responses from the programmers out
there, as I have read their postings in this channel with interest and
would like to hear more of their thoughts.
Before continuing, a few notes/disclaimers:
1) I believe cheating to be an epidemic in both the brick-and-mortar
and online cardrooms.
2) I believe that poker is still a beatable game, online or offline,
if the player is good and CONSIDERABLE caution is exercised.
3) I do not believe that it is possible to make any kind of "living"
in poker without cheating, except for the lucky few - some of whom
post in this channel. Congratulations if you are one of them.
4) I believe that online poker is the poker of the future. I believe
that most if not all of the online venues are well-meaning in that
they would rather rake your money honestly than cheat you out of it
dishonestly.
5) Some of the information in this article might be construed as
giving ideas to cheaters, especially cheaters who know how to write
software. I counter that any programmer worth his salt is already
aware of these methods and a great many others besides. If not, he
would learn them soon enough on his own. And any cheater who doesn't
know how to program would simply ask one of his programming
associates. Ergo, this information will be "new" to one party only:
that subset of poker players who neither program, nor cheat.
---------------------------------------------------
There are two parties potentially interested in bots of this nature:
online poker venues, and online poker players. Accordingly, there are
two classes of bots: venue-sponsored bots, and player-sponsored bots.
My opinion is that the second category of bot is far more common, and
more dangerous to your bankroll.
"I have seen rooms filled with monitors..." wrote a recent RGP
poster, speaking of
Every poker bot in existence can be analyzed according to the standard
"input-processing-output" model. In every case the input is the same:
information about the play of the current hand, including cards dealt
and player actions. This input is then processed by a poker gaming
engine, which is the "brain" capable of playing a winning game. This
"brain" is tasked with delivering a single recommendation to bet,
raise, call, fold, or check. The output portion of the bot then
performs whatever tasks are necessary to inform the client software of
the decision that was made.
INPUT COMPONENT
The task of the input component is to determine the following each
time the bot is faced with a decision:
- The name and stack size of each player at the table.
- The face-up cards dealt to each player, and to the board.
- The face-down cards dealt to the player on whose account the bot is
running.
- The size of the pot.
- The size of any bets made, and who made each bet.
One thing I would like to note is that it is probably impossible for a
player-sponsored bot to "sniff" or otherwise determine what other
player's hole cards are (unless those players decide to show them, of
course). This is because such information is provided by the server
on a need-to-know basis. This means that when Player A receives his
hole cards, the hole cards are only transmitted to Player A. Player B
wouldn't be able to access those hole cards even assuming he could
decrypt the transmission.
It is characteristic of modern operating systems that "windows" are
often used to display textual information. For example, in the
Paradise Poker client, the name of each player sitting at the table,
along with his current stake, are displayed in a standard Win32 window
which any program on the system can query, retrieving whatever text is
displayed. It is trivial to parse this information, and determine
that player "Bob" has "$1540.50". Similarly with the chat/action
window which displays player chat along with information about the
current hand: it is very easy to take the line "bob raises ($40)" and
figure out that bob just put in a $40 raise. This technique can also
be used to retrieve the pot size and the size of any bets.
One method for determining the cards dealt in a particular hand is
what I like to call "dot" or "region" testing. This is a common
image-recognition technique which samples a few choice pixels on the
screen, and from the pixel makeup gleans information about the image
as a whole. For example, by testing 3 pixels in a specific location
at the table, you may be able to determine that the river card was an
Ace of Hearts.
If you have access to the raw card images, a much more efficient
method is available. You simply assign a unique RGB color value to
each card in the deck. Using an image editor you then daub or "mark"
a certain specified region of each card with the indicated color
value. Later, when the bot is playing, the client will use the
modified images. The bot can then sample a single pixel from the
"marked" region of any displayed card, and look at the color of that
pixel to determine the exact rank and suit of the card.
Finally, with a little investigation it may be possible to simply
"pluck" the card values from memory. This will depend on the
implementation of the poker client.
PROCESSING COMPONENT
The poker engine is the brain of the bot. It is responsible for
looking at the available information, and determining which particular
action - calling, folding, raising, checking, or betting - has the
highest expected value for the situation.
Generally and on average, poker bots will make their decisions based
on probability and statistics. The techniques employed may be simple
or quite advanced, but they will generally be limited to "hard" data.
But in order to truly maximize the bot's expected value, the bot must
have an understanding for the soft, fuzzy side of the game as well.
This includes such things as the ability to peg other players as being
on tilt, determining whether the table as a whole is
loose/tight/passive/aggressive, determining the correct time to bluff,
and so forth. For this reason, I believe the development of a winning
poker bot to be more difficult than most players imagine.
Take Wilson Software's Turbo Texas Hold'em as an example. This is
commonly acknowledged as the strongest poker software in the world,
and yet competent players are able to beat it regularly (of course,
for serious players the real value of the product lies in its
simulations). Why should this be the case? Because it is extremely
difficult to imbue a poker bot with a real "feel" for the game, which
is absolutely necessary for defeating the tougher games.
This does not necessarily mean that creating a winning online bot is
unfeasible. It is possible that creating a winning online bot is much
easier, because the bot doesn't have to be generic. That is, you can
optimize the bot to play for a specific betting structure, whereas I
would imagine the Wilson software has to have a great deal of
genericity to allow for players to sharpen their skills across
numerous such structures.
Obviously, once the engine has been written, you want to be able to
reuse it at other venues. The idea is that , in order to exploit a
new venue, you must provide only the input and ouput components. And
the work required for the output component is negligible, therefore,
once the engine has been written, exploiting additional venues is
really just a matter of writing a new input component.
OUTPUT COMPONENT
In almost every client, the user issues commands (such as to raise,
call or fold) by clicking a button. In almost every operating system,
a facility exists to "mimic" or "generate" a mouse-click anywhere on
the screen. This is transparent from the point of view of the client
software; in other words, the client software won't know whether a
particular "click" was the result of a user physically pressing the
mouse button, or
INFILTRATING THE CLIENT
The important thing to remember is that, even assuming you had the
source code for one of the poker clients, it probably wouldn't do you
any "good". The only information you're going to be able to retrieve
is information you would be allowed to see anyway - the board cards,
your own cards, the number of players at the table, and so forth. No
amount of sleuthing around in the process-space of the client
application is going to yield an iota of information that you couldn't
ferret out by simply looking at the screen....UNLESS a particular
poker venue has been careless, or is trying to reduce network traffic.
Even in these cases, rest assured that the data will DEFINITELY be
encrypted, and breaking any serious encryption, let alone SSL (used by
banks, etc), is easier said than done. When people talk about hackers
breaking into systems, this is not what they mean.
That said, the techniques for sneaking code into the address space of
another application have been amply documented by people like Jeff
Richter in his books, and on the internet. The techniques will differ
depending on the specific flavor of Windows operating system running.
If you know enough to get into the process in the first place, you'll
have a good idea of what to do when you're "in".
BOT COLLUSION
If having one bot is good, having twenty or thirty is even better.
After all, if a single bot plays for a week without interruption,
people begin to get suspicious. So intelligent cheats set things up
such that they have multiple accounts at each venue. Since this is
against venue rules, they have to go to various lengths to ensure that
the identity behind each account remains separate.
It is not that difficult to obtain any number of accounts at a
specific venue. One can use a false name or address, or one can use
the name and address of one or more of your friends or fellow
cheaters. Each method has its advantages and its drawbacks. Some
combination of false names and real names is probably ideal, as some
individuals might not want to divulge their real name and address and
yet, for cash-out purposes, a valid receiving name and address might
be desired (especially in this day and age when credit cards are
refusing any and all gambling transactions).
Once the accounts have been obtained, the bots are installed and a
mechanism for connecting to the server is established. Ideally the
physical connection used by each bot is geographically estranged from
the physical connections used by the other bots. This can be done by
relying on friends or co-conspirators in other cities and states, or
by "spoofing" one's connection so that it looks like it's coming from
another region. However, the bots themselves run in a central
location, the geographically isolated or "spoofed" connection points
being proxies which simply forward information between the server and
the centralized "bot depot" which contains the machines actually
running the bots and probably exists in somebody's basement or garage.
Using this mechanism the number of bots that can be put to work is
limited only by one's ability to create new accounts and connections
for those accounts. It is conceivable that over time a pool of
hundreds if not thousands of such bots could be put to some very
diabolical purposes indeed.
CLIENT INQUIRY
While performing research for this post I performed a simple
investigation of the various poker clients, including the Paradise,
Party, Stars, and Planet clients. For this work I used the Visual C++
6.0 IDE along with related tools such as DUMPBIN and DEPENDS.
My initial inquiry revealed that internal symbols have been stripped
from each client, a common industry practise for secure software.
Most of the poker client interfaces are Windows dialog boxes employing
custom bitmaps and painting procedures. Similarly, most of the
information presented on the screen - the flashing circles used to
indicate players, the action buttons, pot and bet sizes, etc, are
implemented as standard windows of one type or another (edit boxes,
buttons, etc).
If you are curious about what some of the administration screens used
by a particular venue look like, simply open that venue's client
software in the Visual C++ IDE (or another such tool) and take a look
at the dialog resources. You will come across a few screens which are
obviously not intended for player use, but nothing too juicy. I did
see something suspicious, however, in one client...I am not sure, and
am not willing to say, what it might be used for. Again, the curious
can investigate for themselves (as this may be a red herring).
All of the investigated clients rely on SSL (secure socket layer) for
encrypted transmissions. Some of the clients shipped with additional
crypto-related DLLs which I was not able to identify. I assume they
are third-party cryptography DLLs from one or another corporation.
From a player standpoint this is very good, as it means the venues are
doing their homework with regard to encryption, as advertised. It is
supremely foolish for players to think hackers are out there
"decrypting" their transmissions when all it takes is a few function
calls to achieve world-class encryption. You are much more likely to
be cheated by collusion.
To Be Continued...
questions, complaints, flames? rambo...@yahoo.com
wow
WoW
"The Other Toke" <rambo...@yahoo.com> wrote in message
news:8009cb01.0210...@posting.google.com...
ParadiseLost
they have the most players and yet most of their games are super tough
to beat.
_________________________________________________________________
Posted using RecPoker.com - http://www.recpoker.com
ParadiseLost
I'm pretty sure Paradise has lots of bots- they have the most
players and yet most of their games suck. Common sense tells you
that the more players in a cardroom, the more fish.
William Coleman
illiterate poker players who are terrified of poker bots.
The writer of this post has provided us with a detailed analysis of problems
associated with constructing a poker bot. He has used classical systems
analysis and design methodology, which is an immediate tipoff that he is
much more than just some technogeek wannabe. His erudition and deep
technical knowledge are clearly visible in every sentence of this brilliant
post.
I guarantee you there is not one programmer on the truepoker technical staff
who is even qualified to talk computers with this man.
--
William Coleman (ramashiva)
home page: www.home.earthlink.net/~ramashiva
IESOUS CHRISTOS THEOU YIOS SOTER (corrupted version)
IESOUS CHRISTOS THEOS YIOS SOTERES (true version)
Sell all your possessions; give the money to the poor; and come, follow me.
-- Jesus Christ
"The Other Toke" <rambo...@yahoo.com> wrote in message
news:8009cb01.0210...@posting.google.com...
D
online sites to detect or prevent the type of information-gathering carried
out by the "input component" of a player-developed bot? That is, of course,
assuming that the sites are really motivated to try. I would think that
significant re-engineering of the client would be necessary, and even then
something like pixel scanning would be tough to prevent or detect. Another
poster suggested that ScoopMonster could easily be disabled, but only
because it exploits some special aspect of the TruePoker client.
Any thoughts?
"The Other Toke" <rambo...@yahoo.com> wrote in message
news:8009cb01.0210...@posting.google.com...
eleaticus
The recent AI thread started me thinking as to how to take a first step to
computer aided play and I immediately came to the obvious solution: monitor
the input & output streams to the modem.
Hah!
As you made clear even before outlining your analysis of actual site
clients.
Eleaticus
"The Other Toke" <rambo...@yahoo.com> wrote in message
news:8009cb01.0210...@posting.google.com...
VladL
service to create state of art bots under personal request? (in difference
to public bots like Scoopmonter?)
Avarice
> Very interesting. To what extent do you think it would be possible for the
> online sites to detect or prevent the type of information-gathering carried
> out by the "input component" of a player-developed bot? That is, of course,
> assuming that the sites are really motivated to try. I would think that
> significant re-engineering of the client would be necessary, and even then
> something like pixel scanning would be tough to prevent or detect. Another
> poster suggested that ScoopMonster could easily be disabled, but only
> because it exploits some special aspect of the TruePoker client.
>
> Any thoughts?
First off, this was an excellent article, and right on the mark. An
online site would have a difficult time detecting this kind of bot,
but could use a few methods to go about it.
They could, of course, check to make sure their files (including the
image files) have not been changed. This would prevent the
pixel-scanning technique as far as changing the image, but of course a
multiple-pixel scan could be used to determine the values of the cards
as they exist in the first place. If I wanted to hide a bot, I
wouldn't change the image files.
Another, and far more plausible technique, would be to heuristically
check the mouse movements of the client. Some sites already record
some mouse movement information, if only for adding random data bits
to their shuffling algorithms (or at least they say they do). A bot
would show up immediately as never moving the mouse, since the bot
would just send MouseClick events to the correct area of the screen.
Therefore, the bot would have to mimic human movements -- a
non-trivial exercise, but one that could probably be done by copying
real mouse movements, and having the bot move the mouse in that way
during play.
In conclusion, a bot that does not alter files, and moves and clicks
the way a human would is going to be indistinguishable from a human
from the server's point of view. This Turing Test for poker should
not be a significant challenge to a good programmer, though it would
definitely take some time to develop. And while a bot to beat WCPs
would be quite a feat, a little something to sit and rake $1BB/hour
out of the $0.5/$1 and $1/$2 games would not be all that challenging,
at least until the room fills up with bots. ;)
--Avarice
Piers Shepperson
> How To Write An Online Poker Bot, Part 1
> ----------------------------------------
>
This post looks fairly sound to me and. It does not appear to differ from
my opion:
There are two components to any poker bot.
1 The interface that communicates with the Client software. This is quite
straightforward to write, and I think any good programmer should be able
to do this in a month or two. As its so easy I expect a lot of people have
done it already, however this is not of much concern because of 2 and 3.
2. The brain that actual determines what action to take. I believe it is
quite difficult to write software to beat the top limit online games. I
expect it will be a several years before such a bot gets created, but
there is an outside chance one is in use now. Certainly I would expect
someone to have one working on the soft lowest limit games by now. Note
once you have a good 'brain' the client plugins for different sites can be
done quite quickly. Online Cardrooms' counter measures could be countered
as and when required.
3. When all is said and done, a winning bot would be just another strong
player or cheat and should be treated as such. It is the weakest players
that drive a poker game so the stronger ones can be largely ignored or
avoided.
4. It will be interesting to see what happens when a decent bot that can
beat the higher limit online games becomes available for public use. I
expect still a few years away but in my (expected) lifetime.
Andrei
> Very interesting. To what extent do you think it would be possible for the
> online sites to detect or prevent the type of information-gathering carried
> out by the "input component" of a player-developed bot? That is, of course,
> assuming that the sites are really motivated to try. I would think that
> significant re-engineering of the client would be necessary, and even then
> something like pixel scanning would be tough to prevent or detect. Another
> poster suggested that ScoopMonster could easily be disabled, but only
> because it exploits some special aspect of the TruePoker client.
I think asking this question kind of misses the point.
The question, IMHO, isn't whether a bot can be prevented or blocked by any
client software. Poker programs are software, no way around it, and given
the software has to draw stuff to the screen, receive information to draw
such things, talk to the operating system, etc., a good programmer/hacker
can find all sorts of ways to make a bot work.
One could use methods of trapping calls, reverse engineering data streams
and such to determine what the information is from the server to make
their bot. And that could be broken everytime the client software changed
protocols, which would be a pain for the bot creator. But one could just
as easily write software that did image analysis of the screen and make
decisions from there.
The question I think people need ot ask themselves is whether a bot can be
given a good set of strategies to ensure making money. That is really what
matters.
And here lies the rub. Can poker be beaten? If you ever heard about the
MIT Black Jack team, you've heard they claim that Blackjack was beatable,
and they succeeded in doing it. They modeled all sorts of data, stats and
probabilities to acheive strategies that allowed them to take over a table
and beat the game. (Mostly collusion based, but even had some strats that
worked well without collusion.) A lot of people thought chess couldn't be
played by a computer, but that wasn't true either.
The question is can a bot be given enough data about statistics,
probabilities, EV, betting and such so that it can develop a play strategy
that MAKES money over time against any set of opponents? Maybe it only
makes one big bet an hour, maybe it only makes half a big bet in an hour.
Since a bot doesn't need to eat, sleep, work, etc., and the fact a
computer is the most patient thing we know, it would easily make money
over the long run with NO EFFORT for the bot creator. Only cost is
electricity and hardware.
And the kicker is this... If a bot CAN do that, that means that poker
really isn't about luck. It probably means most people just aren't patient
enough or good enough to beat the game in the long run. Or know what a
good strategy really is. If this happens, like it did when the chess world
got good programs to play chess, we'll discover more about this game than
we ever cared to know.
But if the bot can't figure out how to win money in the long run given
data analysis... then we'll know poker really is nothing more than game
for those who like to gamble.
My 2 cents.
Andrei
D
With all due respect, your post reveals that you are *completely* missing
the point.
> ... if the bot can't figure out how to win money in the long run given
> data analysis... then we'll know poker really is nothing more than game
> for those who like to gamble.
Um, news flash, we already know the answer to this one. Poker is a skill
game with a big short-term luck component. With a sufficient skill
differential, there will be long-term profit for the better player. Do you
really think that's in doubt? If you do, then you know as much about poker
as you know about "reverse engineering data streams."
"Andrei" <anon...@epeople.com> wrote in message
news:3db71fe4$0$50573$9a6e...@news.newshosting.com...
D
> would be quite a feat, a little something to sit and rake $1BB/hour
> out of the $0.5/$1 and $1/$2 games would not be all that challenging,
While I appreciate and agree with your technical points, on this point I'd
have to go with Toke's statement:
"I believe the development of a winning poker bot to be more difficult than
most players imagine."
Most players think of the easy decisions, like pre-flop hand selection,
playing the nuts aggressively and folding when you miss the flop. They
correctly suppose that a competent programmer could, with a little effort,
write code to make these decisions. However, while making the easy
decisions correctly is necessary for a winning strategy, it is far from
sufficient. There are many, many ambiguous post-flop scenarios that require
the weighing of numerous factors, including opponents' tendencies. This
complexity is why poker takes quite a while to learn. And yes, quite a bit
of this knowledge *is* necessary to beat even a soft low-limit game.
But that's just my opinion, I could be wrong.
"Avarice" <avar...@yahoo.com> wrote in message
news:ad27e4d7.02102...@posting.google.com...
eleaticus
eleaticus
"Andrei" <anon...@epeople.com> wrote in message
news:3db71fe4$0$50573$9a6e...@news.newshosting.com...
William Loughborough
goodbye". Even the best chess-playing machines have rules in their matches that favor them + they get to use references not permitted the live opponent (they had a database of practially all Kasparov's
games and he had none of the computer's).
Everything is simple, nothing is easy and the frequent guesses herein that "'bots to beat soft low-limit games are just around the corner (or already proliferant)" are the usual uninformed speculations
we should all have enough experience to spot as coming from the "or something" solution to really hard problems.
It's not an easy undertaking. I will go long on a machine not winning a meaningful gold bracelet in my lifetime and you can pay my estate.
Love.
Piers Shepperson
incentive to write winning Poke Bots, and hence it will happen.
There is no sufficient financial incentive to send men to Mars, hence it
has not happened, despite the technology being available for years.
jimpepper
William,
FYI,
I have to scroll to the right 3 pages wide to read your posts.
Does anyone else?Does anyone know why?
jim
Paul Phillips
William Loughborough <lov...@gorge.net> wrote:
>But not in mine (I was born in 1926). For over forty years (that I know
>of) I have been told that in six months' time voice recognition software
>would be so effective that I could "kiss my keyboard goodbye".
I forget what the name of this fallacy is, but the fact that somebody
somewhere speculated incorrectly about something to do with computers...
this does not invalidate all speculation about computers. Nor does it
in fact have any bearing on it at all. You've heard other predictions
about computers too, over those forty years, haven't you? And some of
those things happened, didn't they?
>Everything is simple, nothing is easy and the frequent guesses herein
>that "'bots to beat soft low-limit games are just around the corner (or
>already proliferant)" are the usual uninformed speculations
My speculations are more informed than yours are.
>we should all have enough experience to spot as coming from the "or
>something" solution to really hard problems.
I suspect I have solved harder problems than you have. In the field
of programming, by a long distance.
>It's not an easy undertaking. I will go long on a machine not winning a
>meaningful gold bracelet in my lifetime and you can pay my estate.
That may well be true -- there's little reason I can see to welcome
computers into the mainstream of real life poker. But that has nothing
to do with online poker.
Whatever the actual difficulty of writing a bot, there will be extremely
successful bots within a relatively short time frame if there are not
already. And as far as I'm concerned anyone who doesn't agree with this
just doesn't understand some aspect of the issue -- either overestimating
the necessary complexity or underestimating computers and programmers.
We'll let time settle this one, I'm saying this again just to be super
clear that I'm on record, because there is (almost) no doubt in my mind.
--
Paul Phillips | I wonder if jurisdictional issues ever arise between
Everyman | the dream police and the karma police.
Empiricist |
slap pi uphill! |----------* http://www.improving.org/paulp/ *----------
A. Prock
>Whatever the actual difficulty of writing a bot, there will be extremely
>successful bots within a relatively short time frame if there are not
>already.
I can't be certain about the current state of affairs, but
there is strong circumstantial evidence that there was a
fairly successful heads-up bot installed on Paradise Poker
that used the nicks "orson" and "Empty House". This was
more than a year and a half ago.
- Andrew
Jonathan Kaplan.com>
>
>....Whatever the actual difficulty of writing a bot, there will be extremely
>successful bots within a relatively short time frame if there are not
if this is generally true, then there are extremely successful bots right now,
no waiting for any time frame.
>....And as far as I'm concerned anyone who doesn't agree with this
>just doesn't understand some aspect of the issue -- either overestimating
>the necessary complexity or underestimating computers and programmers....
i dont disagree strongly, but i dont agree, either.....extremely successful bots
arent out there now, and it is going to take longer than you think, ...but i do
agree, if i am wrong, then i didnt understand the necessary complexity. i do
think the programming of that extremely successful bot is more complex than you
believe. but then, i know from dealing with the systems department at work,
their time/work estimates are notoriously low....grin
and i also agree, dont underestimate computers and programmers. they do the work
of human gods.
..but usually it takes longer than seven days.
>We'll let time settle this one, I'm saying this again just to be super
>clear that I'm on record, because there is (almost) no doubt in my mind.
>
you are on record, that side of the issue, as strongly as anyone i can think of.
Jonathan
no matter where you go, there you are....
Brandon
> probabilities to acheive strategies that allowed them to take over a table
> and beat the game. (Mostly collusion based, but even had some strats that
> worked well without collusion.)
The accusation of collusion here is out of place. The MIT blackjack
team developed their strategies for winning at blackjack by
statistical analysis and simulation. Collusion between blackjack
players is meaningless when it comes to a winning strategy? Why?
Because all the information is known by all the players (cards are
played face up).
The collusion the MIT blackjack team used was not collusion to WIN,
but collusion to AVOID DETECTION. These are two very different
things. Were it possible to do so, colluding to WIN at blackjack
would be cheating and therefore illegal. What the MIT team did was
not cheating and that is why even when they were caught, they were
merely asked to leave the casinos. Sure, they got roughed up,
threatened, etc. on occasion, but there was really nothing the casinos
could do to them but refuse their business.
Cheers,
B
mensi...@gmail.com
Regards,
Bas
http://www.goedPoker.nl